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ABSTRACT 

This paper presents a study that examined the 
effectiveness of a proposed teaching improvement process, referred to 
as practice-centered inquiry, by using faculty discussion groups 
involving a total of 29 professors, and formal classroom research 
studies, one of which is reported. The study, a collaborative effort 
by some of the professors in designing a formal classroom research 
project, is presented in two parts. The first part examines the 
effectiveness of a faculty discussion group to address the first two 
areas of the practice-centered inquiry continuum (i.e., ranging from 
casual observations to more sustained reflection and activity) . The 
second part focuses on the formal research end of the 
practice-centered inquiry continuum. The results of the study are 
examined from two perspectives. One is from the perspective of the 
formal hypothesis, and the other is from the perspective of meeting 
the objectives of the professors. It is noted that the faculty 
discussion group structure was found to be appropriate in addressing 
the more informal levels of reflection and experimentation, and, 
while only a few classroom studies have been completed to date, he 
process appears to hold promise in contributing to the investigation 
of classroom teaching. Contains 27 references. (GLR) 
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Abstraet 



Faculty discussion groups and formal classroom research studies 
were developed to examine the concept of practice-centered 
inquiry, a proposed teaching improvement pr >cess. A total of 
twenty-nine professors participated in the four discussion groups 
described here. Some of those who participated in the discussion 
groups elected to collaborate in conducting f<>rmal studies in 
their courses; one such study is reported* 
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mtroduotion 

A rmviw of the research on the improvement of teaching in 
higher education, conducted nearly a decade ago, concluded that 
most of the reported findings "reflect only superficial levels of 
experience" (Levinson-Rose and Menges .1981, 419) . Per example, 
it was found that the most frequent measure of the effectiveness 
of workshops, the most common faculty development format, was 
participant satisfaction ratings. Much of the research reviewed 
focused on specific teaching techniques: how to develop a course 
outline; how to organize and present a lecture; how to construct 
a course exam. Generally, these techniques were taught and 
evaluated in isolation without any attempt to fit them into the 
wider context of the teaching-learning process or within the 
professor's existing knowledge of instruction. 

The authors of the review found that research on the 
improvement of teaching in higher education did not represent a 
well-defined area of study, if judged on the basis of having a 
theoretical foundation and incorporating previous research in a 
systematic way. They viewed the exclusive use of quantitative 
methodologies to investigate questions about teaching and 
learning as insufficient. Rather, they recommended qualitative 
methodologies or a combination of approaches which would 
incorporate professors as co-researchers and not merely as 
subjects. 

A survey of the research published in the decade since 
Levinson-Rose and Menges published their review indicates that 
most of the investigations of actual teaching improvement efforts 
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remain doainattd by quantitativo methodologies and generally 
still focus on specific teaching techniques or aptitudes. 
Characteristic of these investigations are those which compare 
methods of giving feedback from course evaluations (Wilson 1986; 
Tiberius, Sackin, Slingerland, Jubas, Bell and Mat low 1989) ; 
evaluate the effectiveness of teaching technique workshops 
through course evaluations (Wentzel 1987) or specific student 
behaviors in the classroom (Mahler and Benor 1984); or, analyze 
which teacher behaviors yield higher course evaluation ratings 
(Cranton and Hillgartner 1981) . These studies offer a valuable 
contribution to the field, but they do little toward building a 
comprehensive understanding of the teaching and learning process 
in higher education. 

In the last ten years, some attempts have been made to 
propose a theoretical framework for studying the process of 
teaching improvement in higher education. Kozma (1985) applied 
concepts from the study of innovation in complex organizations to 
instructional innovation in universities. A grounded theory of 
instructional innovation was then constructed based on the data 
from the analysis of four institutional projects. Two additional 
institutional projects which have been examined by another 
researcher corroborate these findings (Elrick 1990) . Emerging 
patterns indicated that changing a teaching approach is a very 
personal process, one which is more likely to occur and continue 
if the change is perceived as meeting some identified need and if 
the change process is supported by peer group interaction. The 
most consistent finding indicated that new teaching behaviors 



evolve fro» past practice. New practices are generally not 
considered and rejected based on their merit to enhance learning 
or to satisfy an identified learning need, but on their closeness 
to previous practice. 

Another attempt to provide a theoretical framework for 
tejiching improvement in higher education depicts the process of 
improvement as a developmental model. Pour levels or stages are 
identified, each characterized by certain perceptions of teaching 
and learning and associated teaching practices. At the first and 
least developed stage, teaching is viewed by the instructor as 
telling; presenting information as facts. Development continues 
until, at the fourth stage, teaching has evolved into a "complex 
interaction of students, content and teacher actions" (Sherman, 
Armistead, Fowler, Barksdale and Reif 1987, 78-79). it is 
suggested that movement between the stages is encouraged by a 
number of considerations, including an opportunity for structured 
reflection, sufficient time to make shifts in thinking and 
action, considerable involvement, moderate levels of challenge 
and peer support and encouragement. This work, consistent with 
that described above, emphasizes the personal nature of 
instructional change. 

Given the relative dearth of attention to theoretical issues 
in the study of teaching and learning in higher education, one 
cannot ignore the interesting theoretical developments at the 
elementary and secondary education levels, of particular 
interest here is the area of research referred to as classroom 
processes and cognitive science research (Shulroan 1986a). 
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Includ«d in thi« area of research are the examination of 
claasrooB settinge, classroom interactions, teacher cognition and 
decision making. More recently, the role of knowledge in 
learning has been investigated, primarily from the perspective of 
current cognitive theory which emphasizes the process of 
knowledge construction, the role of prior knowledge and the 
learning context (ResnicK I98i; 1989). Perhaps most applicable 
to higher education, where professors generally think of 
themselves first as subject, cuatter experts and researchers and 
then as teachers, is the proposed study of the relationship 
between teachers' subject maimer understanding and their 
approaches to instruction Three categories of teacher knowledge 
are distinguished: subject matter content knowledge, pedagogical 
content knowledge and cum'culav knowledge. Principal questions 
which have been suggested -t^ -fvine the investigation in this area 
are: "What are the domavn« and rategories of content knowledge in 
the minds of teachers? How. for example, are content knowledge 
and general pedagogical knowledge related? In which forms are 
the domains and categories of knowledge represented in the minds 
of teachers? What are proaiting ways of enhancing acquisition 
and developaent of such knowledge?" (Shulman 1986b, 9) . These 
seem to be questions which could be equally appropriate in 
framing the investigation of teaching and learning in higher 
education. 

In fact, in the last decade, there has been some attempt to 
explore these kinds of questions in the context of higher 
education. Examples are: descriptions of professors thought 
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procMMS concerning course planning and monitoring (Powell and 
Shanker 1983) ; the development of methods for exploring course 
content (Donald 1983); the examination of graduate teaching 
assistants' implicit theories of teaching and learning (Menges 
and Rando 1989); the evaluation of a mentoring project for new 
faculty (Boice and Turner 1989) ; and, an investigation of the 
relationship of professors' personal methods of research and 
knowledge development and their instructional methods (Shore, 
Pinker and Bates 1990) . 

The study described here seeks to contribute to this 
literature by examining the effectiveness of a proposed teaching 
improvement process which has been referred to as practice- 
centered inquiry (Van Note chism and Sanders 1986) or as 
collaborative action teaching research (Schratz 1990). These 
proposed structures for teaching improvement are based on the 
premise that a large part of a professor's knowledge about 
teaching evolves from reflection and experimentation. The 
authors describe the practice-centered inquiry process as a 
"continuum of activities" (Van Note Chism and Sanders 1986, 58). 
At one end of the continuum are observations, realizations and 
questions that arise during the process of teaching or soon 
after. The center of the continuum is characterized by more 
sustained periods of reflection which may lead to a change in 
some aspect of one's teaching approach. At the other end of the 
continuum, practice-centered inquiry becomes formal research. 

The study described here is presented in two parts. The 
first part examines the effectiveness of a faculty discussion 
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group to addrosfl tho first two areas of the practice-centered 
inquiry continuuM (i.e., ranging ; rom casual observations to more 
sustained reflection and activity) . The faculty discussion groups 
were designed to be consistent with both the emerging theoretical 
frameworks of teaching improvement in higher education and 
current cognitive science theories of learning and teaching. 
That is, they were designed to foster reflection and informal 
experimentation by professors in an interpersonal and 
intellectual atmosphere focusing on the relationship between 
subject matter content, the process of learning, pedagogical 
decisions and teaching behaviors. 

The second part of this study focuses on the formal research 
&nd of the practice-centered inquiry continuum. Some of the 
professors who participated in the discussion groups chose to 
collaborate in the design of a formal classroom research project; 
one such study is described. 

Faculty Discussion Groups 
Each semester for the last two and a half years, faculty 
discussion groups focusing on teaching have been organized. 
Participation is voluntary and advertising is done through campus 
mail and the campus newspaper. Past participants have represented 
a variety of academic and professional departments including: 
Animal Science, Anthropology, Business Management, Chemistry, 
Economics, Educational Psychology, Family Medicine, Geography, 
Geology, Internal Medicine, Mathematics, Metallurgical 
Engineering, Nursing, Pharmacy, Physical and Occupational 
Therapy, Physiology, Political Science, and Psychiatry. The 
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groups havs rangad in size from 6-12 participants with years of 
teaching experience ranging from one to twenty, a total of 
twenty-nine individuals participated in the four groups described 
here. 

The discussion group participants decide how often they will 
meet. Three of the four groups described here chose to meet 
weekly for two hours over one semester. The fourth group met bi- 
weekly over the entire academic year. An effort is made to 
create an informal, comfortable setting for the discussions. 
This seems to be an important element as written comments from 
participants have made specific reference to the benefits of the 
"informal setting", "round-table arrangement", "optimal group 
size", "comfortable manner", "conduciveness to discussion" and 
"relaxed, yet constructive atmosphere". 

The content of the discussion groups is based on information 
provided by participants. During the first meeting they are 
asked to fill out a form which addresses, among other things, 
their teaching experiences and the ideas or concerns they would 
like to discuss. Responses have ranged from questions about 
basic teaching procedures (e.g., "how to set and evaluate 
assignments"/ -How to determine the goals and structure of my 
course") to complex teaching issues which present an on-going 
challenge for even the most successful instructor (e.g., "How to 
help students develop understanding and make links between theory 
and what they see in the lab or real life"; "not being able to 
get the students to understand what I have learned, digested and 
integrated over the years"). From these comments, a tentative 



diacu«»ion agenda is developed by the discussion group leader and 
presented as a general direction for proceeding. 

The following section describes the content of the 
discussions held by four groups over a two year period. Each 
group met for twelve to fourteen sessions. The description is 
taken from the notes of the first author, recorded after each 
session. The first author served as the discussion group leader 
for all four groups. Additional information was provided by the 
initial assessment of interest, a subjective evaluation of 
learning and participants' individual written evaluations 
completed at the last discussion session. The general discussion 
topics considered by all four groups included: 1) analysis of the 
subject matter; 2) analysis of the learning task; 3) matching of 
teaching methods and learning expectations; and, 4) evaluation of 
learning and teaching. Each of these will be discussed in turn. 
Analysis of the Su bject Matter 

One of the first topics proposed for discussion was the 
structure of a professor's subject matter. Most of the 
discussion croup participants had never thought about the 
structure of subject matter within their disciplines or in 
comparison to other disciplines. The knowledge structure, a 
method of "concept mapping," was introduced (See Figure 1). This 
method has been used to gather data about the structure of 
knowledge across disciplines (Donald 1983). In the discussion 
sessions, initial practice with the method was accomplished by 
working together to construct a knowledge structure for one of 
the courses being taught by a group member. For example, in one 
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group, a courst from Animal Sciences concerning the function of 
food fro» e nutritional, social and political perspective 
provided an ideal practice possibility because of other 
participants' general familiarity and interest. Knowledge 
structures seem to have had general appeal, as evidenced by the 
fact that all discussion group participants constructed a 
knowledge structure for one of their own courses and explained it 
to their faculty discussion group in the following session. 

The resulting sessions always created much interest. The 
opportunity to see the way. that others approached structuring 
their subject matter and reflecting their own knowledge proved 
fascinating. Some brainstor^ned all the concepts of the course and 
then narrowed them down to the major ones, others went to their 
textbooks, course notes, course outlines, etc. before decidin.7 on 
the major concepts. There was much discussion about the 
difference between topics and concepts and how to movB from 
thinking about topics on a course outline to thinking about 
concepts that reflect the development of knowledge. All 
participants indicated that the process had required much 
thought. One professor discovered that he wanted to teach 
process rather than information, so his knowledge structure 
represented an analysis of a process which was then to be applied 
to various hunter-gatherer societies studied in his course. 

Further discussion led to the advantages and disadvantages of 
sharing with students knowledge structures developed by 
professors, it was generally agreed that this would depend on the 
subject matter and the course level. One group brainstormed the 
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fol loving list of reasons to consldor sharing concept naps with 
students: 1) to provide a more specific idea of course content; 
2) to provide a view of the professor's mental organization of 
the content and the course; 3) to provide an example of how one 
expert, after years of intellectual endeavour in this field, 
would represent the knowledge to be gained in the course; and 
4) to provide an incentive for students to think about the course 
content in a cohesive, meaningful way. Another group felt that 
the students would also benefit from an explanation of why the 
professor was motivated to construct a knowledge structure, what 
was the process of constructing it, what impact it had on his or 
her own thinking and why he/she thought it beneficial to share 
with students. 

Several of the participants did share their knowledge 
structures with their students soon after constructing them. Two 
of them reported that some students had been prompted to compare 
their own mental maps to that of the professor. For other 
students, it seemed to prompt an "Aha!" effect: "so that's how it 
fits together I** One professor described his students as seeming 
to be confused, but admitted he had provided very little 
explanation. Two of the professors reported that they had made 
changes in their knowledge structures based on the comments of 
their students. 
Analysis of tha Learning Task 

The development of the knowledge structures which involved 
consideration of major course concepts and their 
interrelationships, easily led to a discussion of the kinds of 



learning sicpsctations professors held for students. The 
knowledge structure exercise was a clear demonstration to the 
professors that most of them expected much more from their 
students than simply the retention of information. The complex 
relationships between concepts indicated by their knowledge 
structures generally required more sophisticated levels of 
thinking and learning. 

The Taxonomy of Learning Objectives: Cognitive Domain (Bloom 
1956) , was introduced to provide a vocabulary and structure for 
discussing different types of learning. The cognitive domain 
includes knowledge, described as the simple recall of 
information, and t^e higher cognitive skills of comprehension, 
application, analysis, synthesis and evaluation. Kemp (1985, 
84) further described each of these levels using descriptive 
verbs. For example, verbs to describe analysis include: 
calculate, categorize, discriminate, and differentiate. One 
professor provided an example which illustrated the notion of 
higher cognitive skiMs in his course: He was pleased to note 
from his midterm exams that students had managed to retain the 
most important information from the lectures and the vast amount 
of required reading, but he was disappointed that there was not 
more evidence of either the integration of information from 
various sources or the ability to apply this in analyzing novel 
situations. He considered these abilities to be essential for 
anyone wishing to continue in his discipline of political 
science. 
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Th« taxonomy was not accepted by any of the groups at face 
value. The discussion always turned to whether or not one could 
discuss knowledge in this way and whether or not the categories 
really met the definition for a taxonomy, with skills listed in 
sequence of difficulty or sophistication. In addition to the 
cognitive domain of learning, the affective and psychomotor 
domains of learning were also introduced and discussed. These 
were of particular interest to those teaching in clinical areas, 
for example, in the fostering and evaluating of interpersonal 
skills, values and attitudes. 

Participants were encouraged to use their knowledge 
structures to begin to articulate the learning they expected from 
their students. In all groups, there were some individuals for 
whom instructional or behavioral objectives held negative 
connotations. Many had been forced to write objectives out of 
context and according to a 'specific formula; the exact purpose of 
such an exercise had often been unclear to them. Some persisted 
with these negative attitudes and others decided there was some 
benefit in trying to clearly describe the expected learning. A 
total of fifteen of the twenty-nine participants in the four 
groups developed learning expectations based on their knowledge 
structures and presented them to their group for feedback. 

Although the specific discussions varied considerably, in all 
groups over fifty percent of the meetings were spent in analyzing 
the subject matter and the expected learning. This included 
developing knowledge structures, discussing the notion of various 
types or levels of learning and determining learning 
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expectations. Several participants renarksd thAt they had begun, 
for the firet time, to ?ook at teaching from the perspective of 
learning rather than as the organization and presentation of 
information. 

Matching TMChinq Methods and T.aarnincf Rvp >^^ ^^^ ^ „^ 

It was clear from the initial information collected from all 
participants that many had specific questions and anxieties about 
instructional methods and classroom formats (see Appendix 1) . 
Some also stated that they were interested in innovative methods 
of teaching. One more senior Tnathematics professor stated that 
he had taught many different courses, both large and small, but 
he always taught them in the same way. Something about this made 
him feel uncomfortable, but he did not know what to do about it. 

Instead of simply describing various teaching methods and 
practicing them, the idea was introduced of matching teaching 
methods to the specific subject matter and the expected learning. 
To begin discussion, two charts were handed out which organized 
teaching methods according to different kinds or levels of 
learning (Weston and Cranton 1986). m all groups, several 
minutes of silence followed as the participants reviewed these. 
Several stated that they had never thought of teaching in this 
way. Selecting instructional strategies to specifically support 
certain kinds of learning was a new perspective. Another 
commented that it was obvious that one could not learn to ride a 
bicycle, for example, with only a verbal explanation, but they 
had never thought about the function of "practice" in the 
improvement of intellectual skills such as analysis, synthesis, 
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and evaluation. In all the groups, a few people immediately 
jumped to discussing the possible mismatch between lecturing, the 
predominant teaching method in higher education, and the kinds of 
learning expectations we had discussed in earlier sessions. 
Others were skeptical of the general idea that teaching methods 
and level of learning were closely related. In an end-of-the 
semester evaluation, the following question was asked: We have 
discussed the notion of higher level cognitive learning (e.g., 
abilities of application, analysis, synthesis, evaluation, etc.). 
I am curious to know: l) yoir thinking concerning whether 
teaching can facilitate i:hi5 /(*nd of learning, and 2) what place 
these ideas have in univefs-ity level programs. The following is 
a sample of the responses 

There needs to be nor* -focus on the teaching of 
thinking. Institutes- » cf- higher education are 
definitely the place j^/ levelopment of intellectual 
skills but there does fK^ =:oem to be any conscious 
effort to consider it; at a teaching issue. 

I think teaching can facilitate the development of 
these intellectual skiILs a) the courses are 
designed such that eaphasis is placed on these skills, 
and b) the students ar« initially made aware of the 
importance of these skills and that their (the 
students') progress wHl be evaluated on the basis of 
their ability to master them. These skills should 
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ideally b« th« central focus /overall goal of most 
disciplines. 
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There seens an urgent need to reconsider the role of 
the student and teacher in the university setting. Far 
too many courses still seem to treat students as 
nothing more than 'information-receptacles', i am 
quite convinced that different teaching techniques can 
help a great deal to develop the intellectual skills 
mentioned - and at the same time develop the students' 
sense of confidence, initiative and responsibility. 

To a certain extent but students are primarily 

responsible for their learning at this level. 

In discussing specific instructional strategies, familiar 
methods were addressed first, namely the lecture and lectures 
combined with discussion. Many were interested in using more 
discussion in their courses, but most felt uncomfortable with it. 
Some of the anxieties they described about conducting class 
discussion included: 

- What if I ask a question and no one responds? 

- What if someone asks a question I cannot answer? 

- What if students t^^et bored? 

- What if one or two students dominate the discussion? 

- I am worried that the discussion will not stay on topic. 
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- Th«r« is so much information to cover, I hate to take time 
for discussion. 

Most participants thought of class discussion as the 
professor posing a question to the entire class or a student 
asking the professor a question in front of the entire group. A 
successful discussion was pictured as one in which everyone is 
listening and, one at a time, questions are asked and answered. 
The idea was introduced of dividing students into small groups 
for discussions and providing some structure depending on the 
instructional purpose for the discussion. This approach created 
interest, but most could not irunediately picture themselves doing 
it. 

One of the activities involved providing participants with 
short handouts describing various discussion methods and 
arrangements. After reading them, each participant made notes 
about how they might incorporate this method or arrangement into 
their courses or, if not, why not. Their notes were then 
discussed in pairs and each pair described their ideas to the 
full group. In each of the four faculoy discussion groups, a few 
participants tried using at least one of these methods in their 
own course during that week and then returned to tell the group 
about it. Everyone seemed to become involved in discussing the 
possibilities of structured discussion in their courses, yet not 
everyone was convinced that they had the skills to do it. 

An interesting observation at this point is that after 
several discussion sessions, participants became quite supportive 
of each other in terms of providing encouragement to experiment 
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with cartaln idaas in their courses. Often, the group came up 
with ideas about how to try something at a level one could be 
comfortable with. 

Information about other instructional strategies (e.g. , 
inquiry groups, peer critique, project methods, laboratory 
teaching, etc.) was handled in the same way: a) participants 
selected a short reading concerning an instructional strategy; 

b) participants explained the strategy to others; and, 

c) participants discussed the appropriateness of the strategy to 
their own teaching tasks. In this way, participants were 
presented with many ideas in a meaningful way and were provided 
with practice in selecting instructional strategies based on the 
structure of their subject matter and learning expectations. 
Evaluation of Learning and Teaching 

There are many issues surrounding exams and grading for 
most professors. All participants in these faculty discussion 
groups agreed that there was considerable faculty interchange 
regarding exams and grading, even though they frequently 
commented that they rarely, if ever, heard or participated in 
conversations about teaching problems and approaches. 

Participants (questioned whether the purpose of exams and 
assignments actually was the evaluation of learning. It was 
generally agreed that most professors viewed exams and 
assignments not as a way to evaluate learning, but rather as a 
way to separate out or rate students against one another. Poor 
exam scores, they thought, were hardly ever taken as an 
indication of poor teaching. If students did too well on an 



exan, th« ttst was seen as inadequate, but if students did 
poorly, then students had not prepared. According to 
participants, these views constitute the dominant attitudes of 
professors, in general. The discussions about exams and grading 
often seemed to dead-end and it was the participants who taught 
in clinical areas who usually pulled the discussion back to 
considering what type of learning was being evaluated. As one 
person from medicine stated: "We are acutely aware of the need to 
realistically evaluate knowledge and performance because we are 
educating future practitioners". 

Interest was expressed about evaluation methods other than 
written exams. One person suggested that the problem with oral 
examinations was seen to be the loose organization, where neither 
students nor professors were really clear as to the purpose of 
the examination. In two of the groups, there were enough people 
from various medical programs to be able to create a separate 
group to discuss the evaluation of clinical skills, an area where 
written exams are plainly inappropriate. The discussion group 
leader provided them with a reference which addresses the 
construction and interpretation of appropriate evaluation methods 
(Evaluating Complex student T.e;>>-nipq^ Cranton 1982). 

Student problems concerning cheating and plagiarism were 
major concerns and, as one participant pointed out, perhaps 
larger than was justified considering the actual number of 
students involved, others disagreed that the problem was that 
small. 



20 

It was clsar from the discussions that many professors have 
an attitude of *'me versus them" when thinking about examinations 
and assignments and less often view It as a true evaluation of 
expected learning. Many participants voiced the concern that 
sharing learning expectations with students and baslrq exams on 
them constitutes ** spoon- feeding'* and that too many students would 
do too well. Others countered with the idea that higher quality 
teaching should produce more learning overall but that there 
should still be a distribution of marks. As with most of the 
other discussions, no specific consensus was reached, but 
discussions were involving and provided a view of various 
perspectives. 

PaiTticipant Evaluations of the Discussion Groups 

Participants were asked to complete an end-of-the semester 
evaluation about the composition of the group and the structure 
and format of the sessions. Regarding group composition, all 
participants stated that there were advantages to having a multi- 
disciplinary group. One person stated that it was beneficial to 
'*be able to Identify with both the successes and failures of your 
peers." Another stated that «*I also learned a lot about the 
startllngi differences in the subject matter demands in different 
faculties." A third commented that "I may not have felt as free 
to express myself If the group had been composed of only faculty 
from my own department." This comment may have some relationship 
to the frequent observation of several participants that faculty 
do not talk about teaching and questions about it are answered 
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vary sp«cif ically without room for much conversation about 
teaching . 

When asked to compare the structure and format of the 
discussion sessions with that of workshops, the responses were 
consistently in favour of the discussion format. Examples of 
some representative responses follow: 

A discussion group is better. I think that knowledge 
has to be explored and applied by the individual, and 
any more formalized or unified approach would probably 
tend to impose more of a common 'answer' to specific 
problems . 

I find the weekly discussion format more useful in 
that, in the long run, we probably cover more material 
than in a day long, or weekend workshop/ seminar. Also, 
the material is easier to assimilate in smaller doses. 



The weekly group sessions are much better than a 
seminar. The exchange of ideas is uninhibited and 
personal rapport is fostered. 
One person stated: 

Now that I know some of the questions, i w. uld prefer 
to have a workshop directly focused on teaching 
techniques. 



Twenty-five of the twenty-nine participants felt that the 
sessions had completely met their expectations. Two added they 
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would h«v« likttd to sp^nd more tin* on evaluation methods, one 
wished acre time had been spent on "problems we were facing in 
the classroom and how we were (or were not) solving them." 

Another set of qu^itions asked participants if they planned 
to change or adapt anything about their teaching aad if they 
desired further support in doing so. Twenty-one of the twenty- 
nine participants stated they would use (or had already used) the 
knowledge structure method in planning courses and would like to 
experiment in using it with students, sixteen planned to 
incorporate learning outcomes in the planning of their courses. 
Five were going to develop reading guides based on assigned 
readings and planned to use them as the basis for class 
discussion. Twenty-two planned to (or had already) experimented 
with various class discussion methods and the lecture/discussion 
combination method. 

Thirteen of the twenty-nine participants wanted some kind of 
further support. Two of the professors continued to meet 
individually with the discussion group leader to plan new 
courses; another two to develop new course materials. Three 
participated in the faculty discussion group again during the 
next semester. Five wanted to attend a continuation support 
group. One borrowed more reading materials and two requested 
help in identifying highly rated professors in the university so 
that they could sit in on some of their classes. Five professors 
from one group were interested in getting more professors 
involved in the faculty discussion groups and wanted to remain 
involved themselves. Three of these individuals have made 
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prMMtatlons for th« last two years at the annual orientation 
for new faculty. Plans are now being discussed to begin some 
kind of peer consultation/mentoring program. 

Participants were also asked if they would be interested in 
collaboratively designing a formal study in one of their courses, 
based on changes they would like to make. Several have been 
interested in formal classroom research studies, but available 
resources have allowed only three studies to be completed and two 
more to be planned. These are discussed in the following 
section. 

Formal Classroom Research studies 

Introduction 

The design of each formal classroom research study is based 
on the questions and concerns of the professor. An initial 
interview is done to try to understand as completely as possible 
the context of the concerns expressed. The appropriateness of 
possible instructional interventions is then considered based on 
the match between the professor's questions and concerns and the 
comfortableness of the professor with implementing them. A 
literature review is then conducted to inform the design of the 
study and to see how a contribution can be made to the current 
literature. One of these studies is described in some detail in 
the following paragraphs. 

The professor of the course, a participant in a faculty 
discussion group the previous year, had already implemented a few 
changes in the way he approached teaching. In this study, he was 
interested in addressing three areas he felt important in 



taaching his large introductory course: i) his own lack of 
enthusiara in teaching a large (n-209) introductory course; 2) 
what he felt to be an "underuse" of his teaching assistants; and, 
3) the level of learning evidenced by his students. The 
professor thought his lack of enthusiasm for teaching this 
particular course came both from feeling relatively isolated from 
the students and from having to lecture on information from the 
textbook instead of supplementary information which interested 
him more. He was using his teaching assistants only for grading. 
He had, in a previous semester, asked each teaching assistant to 
be responsible for one class lecture, but he had not been 
satisfied with the results. Finally, he was not happy with the 
level of learning he felt was being achieved by his students. He 
felt that they were not integrating information from various 
sources nor were they able to synthesize and apply the basic 
conceptual foundations of the course. 

To address these concerns, it was decided that the format of 
the course would be changed to include one weekly tutorial or 
discussion section of about forty students each and only one, 
instead of two, large group lecture classes. Reading guides 
based on the textbooks were developed to guide students in 
finding and synthesizing the most important information in the 
readings. All students received the reading guides so that 
exposure to the text material would be equal for everyone. The 
reading guides formed the basis for discussion in the smaller 
discussion sections and the structure of the questions served to 
encourage the level of learning expected. They also freed the 
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profMiior from lecturing on the text material, thus allowing him 
to includs nore supplementary information. The tutorial sections 
were taught by the professor and the two graduate teaching 
assistants, providing both n^ore individual student ':ontact and a 
structured teaching situation for the teaching assistants. The 
professor participated in th« development of all the instruments 
and activities, as he did in all facets of the study. He and the 
two graduate teaching astittants also participated in weekly 
research team meetings wr.er* problems, concerns and ideas were 
discussed. 

Concept mapping was chosen as the experimental variable. 
Concept mapping is a visual representation of relevant concepts 
and the relations amonq t^)em. Although the research on the 
effectiveness of this sttafe^*^ :s not yet conclusive, there is 
some evidence that its uS€ in the university classroom can 
improve both the amount and (dvel of learning accomplished 
(Okebukola and Judge 1988. hlLoV^< 1990; Mahler, Hoz, Fischl, Tov- 
ly and Lernau 1991). Bas«] on the information from existing 
research, three treatment 3roups were established. These and 
other details are discust«4 \f\ the following section. 

Sample . Undergraduate s^u:lents (ii=209) enrolled in an 
introductory Anthropology Course offered in the Fall session of 
1990 participated in this ^tuJy. Results from a preliminary 
questionnaire administered tro both students enrolled in the 
course one year prior to this study and to those who served as 
subjects in this study indicated that students in this course 
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t«nd to b« hoBogansous in age (mean age of 19 years) and in their 
second year of university. There are approximately equal n.riabers 
of nales and females and most enroll in the course because of 
interest in the course content. Students represent a diversity 
of academic backgrounds. In the two semesters surveyed, eleven 
academic disciplines were represented. All students in the 
course were involved in the research study, as participation was 
an integral part of the course. 

Ingtrumwtflti<?n- Three instruments were developed and 
utilized: 1) a pre-post test of conceptual knowledge; 2) a 
demographic questionnaire; and. 3) an informal assessment of 
student attitudes about the effectiveness of various course 
components . 

The test of conceptual knowledge was administered to all 
students during the first week of instruction and again during 
the last week of instruction. To construct the test of 
conceptual knowledge, a list of major concepts was compiled from 
the required readings for the course. The concepts were 
operationally defined as definitions, theories, sequences of 
procedures, or other units of information. To further refine the 
list, the professor of the course selected only those concepts he 
considered to be a reflection of the overall learning task and he 
added other concepts he feit were missing. This selection 
process continued until 30 concepts were selected as the most 
important or key course concepts. From the 30 key concepts, ten 
were randomly selected for each of 3 open-ended test forms. As a 
result of the random selection, most of the concepts appeared on 
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mor« than on« form, while one or two were never selected. The 
three tormm of the test were randomly distributed among students. 
There was, therefore, a random chance that students were given 
the same form of the conceptual knowledge test during the pre- 
and post-test administrations. Written instructions on each test 
requested students to define and explain the concepts to the best 
of their ability within twenty minutes. 

The tests were scored by comparing each concept definition 
with the professor's. The scores ranged from zero (wrong or no 
response) to three (complete definition) . In order to establish 
inter-scorer reliability, the two graduate teach:Lng assistants 
for the course were asked to score ten tests randomly chosen from 
those collected. The agreement rate obtained in each case was 
92%. In comparing the teaching assistants' scores to one 
another, an agreement rate of 99% was obtained. 

In addition, concepts given a score of zero were analyzed 
descriptively. If an inaccurate explanation of tha concept was 
given, it was marked as a misconception. The number of 
misconceptions was tallied for each subject. 

A demographic questionnaire was administered at the 
beginning of the semester to gather information on sex, age, 
academic major, previous anthropology courses taken and years in 
university. The informal assessment of student attitudes was 
administered at the end of the semester. 

For one semester, students attended an hour-and-a-half 
lecture in a large lecture hall, and an hour-long discussion 
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session ones a vssk in one of six smaller classroom settings. 
The discussion sections constituted the three treatment groups. 
In order to maintain an average number of 30 to 40 students in 
each discussion session, students were randomly assigned to the 
six sessions. Thus, each treatment group included two discussion 
sections. 

The hypothesis of this study was that students with a more 
varied exposure to concept mapping would make greater gains in 
conceptual knowledge and exhibit fewer instances of 
misconception. All students were given the same basic reading 
guide that was developed to focus on the most important 
information in the two required textbooks, it was the inclusion 
of concept mapping activities in both the reading guide and the 
discussion sessions that distinguished the three treatment 
groups. Conditions for each treatment group are described as 
follows: 

In the first group, students (n=57) were given a reading 
guide that providing them with questions and examples from the 
text material. In the second group, students (n=54) received the 
same reading guide but with the addition of concept mapping 
questions inserted at various points. These students were asked 
to construct concept maps to show their understanding of the 
interrelationships among important course concepts. No time was 
allotted during the hour long discussion session for discussion 
or feedback on their concept maps, but if students raised a 
question, it was answered by the instructor, in the third group, 
students (n-46) received the reading guide with the same inserted 
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concept mapping questions as group two, but for this group, tine 
was allotted for students to discuss each others' concept maps in 
peer groups and to receive feedback from the instructor as tine 
permitted. 

A true control group was not established because the 
professor felt that all students should at least be introduced to 
the basics of concept mapping as a learning strategy. He 
therefore gave one lecture to all the students in the course. In 
this lecture, he explained the purpose of concept mapping as a 
learning strategy and suggested a procedure for creatin>j concept 
maps. He then presented some examples. 

The professor and two graduate teaching assistants were the 
instrvictors of the discussion sessions. Each taught two 
sections. In order to roinimi?.e the possible effects of teacher 
bias, the instructors rotated among groups every four weeks, 
adapting their teaching methods to the requirements of each of 
the three groups. This also had the benefit of providing an 
opportunity for the professor to form his own perceptions of the 
extent to which both the reading guides and the concept mapping 
activities motivated students and helped them to meet the 
learning expectations of the course. 
Results 

The analysis was based on the responses of 124 students from 
whom complete data sets were collected. Multivariate Analysis of 
Variance indicated that pre- and post-test scores were 
significantly different across all three groups (F=«153.66, 
p<.001). The means and standard deviations of the pre- and post- 
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test scores for each group are presented in Table i. Group three 
made the largest gains, followed by group two and group one, 
respectively. However, significant differences also appeared 
among pre-test scores, despite random assignment. There also 
appears to be a ceiling effect among the mean post-test scores 
for all groups. 



Insert Table i and Figure 2 about here 



The numbers of misconceptions on pre- and post-test scores 
were tallied for each group, students in group three, who 
received the most varied exposure to concept mapping, showed the 
largest decrease in their percentage of misconceptions from pre- 
to post-testing, followed by students in groups two and one, 
respectively (Table 2). There were no significant differences on 
course exam scores for the three groups, either as analyzed by 
composite scores or broken down by objective and essay questions. 



Insert Table 2 and Figure 3 about here 



The analysis of the relationship between demographic 
variables and pre- and post -test scores of conceptual knowledge, 
analyzed using Multivariate Analysis of Variance, showed that 
older students had higher n>ean pre-test scores (F«10.12, fi<.05), 
as did those who had taken previous course work in Anthropology 
(F=4.46, E<.05). No significant effects of sex, academic year or 
academic major were indicated (e>.05). 
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An analysis of tha Informal assassmant of student attitudas 
Indlcatad that 76% of the students found the reading guides 
either very helpful or quite helpful in understanding the 
textbook material. When asked hov useful concept mapping was as 
a learning strategy, 38% of the students indicated that they had 
found it somewhat to very useful. To preserve anonymity, the 
attitude assessment coald not be analyzed by treatment group. 

The results of this study must be discussed from two 
perspectives. One is from the perspective of the formal 
hypothesis and the other is from the perspective of meeting the 
objectives of the professor. Each will be discussed in turn. 

Students significantly increased their conceptual knowledge 
from pre- to post-testing in all groups. The reading guide, 
common to all groups, was developed from the identif i'" J! '•ajor 
course concepts and was designed to direct students to .levant 
text information. An effort was made to construct questions 
which required students to be able to apply knowledge to domain 
specific problems and to synthesize important information 
presented. The reading guides seemed to be readily received by 
students with 76% reijorting them to be quite helpful or very 
helpful. The reading guides may have been sufficient to aid 
students with integration and synthesis of material and therefore 
lessened the effect of the concept mapping intervention. A 
closer examination of how to most effectively use reading guides 
to facilitate learning may prove to be an interesting 
investigation. 

:i3 
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Post-ttst scores of conceptual knowledge indicated a ceiling 
effect. Ths type of test used to assess conceptual knowledge may 
have been limited in its ability to assess the depth and 
structure of a student's conceptual knowledge. 

An analysis of student's misconceptions indicated that 
concept mapping may have aided students in correcting 
misconceptions, in group three, where students had the 
opportunity to discuss their concept maps with each other and the 
instructor, the percentage of corrected misconceptions was higher 
than in the other two groups. 

Students were advised that they would not be graded on 
concept mapping questions, nor would they be asked to use them on 
an exam. Concept mapping was presented only as a potentially 
useful learning strategy. Yet, by far the majority of the 
students in the two concept mapping groups constructed maps each 
week, even though only those in group three had the opportunity 
to discuss their responses. This consistent completion of the 
concept maps suggests that students found them useful or 
interesting, or both, and that it is an appropriate instructional 
intervention for coursework at this level. Further investigation 
might pursue the usefulness of concept mapping in other content 
domains and include variables which focus on individual 
differences in the usefulness of this learning strategy. 

From the perspective of meeting the objectives of the course 
professor, both personal views and continued practice were noted. 
The professor stated that he enjoyed lecturing on information of 
interest to him, instead of feeling bound to cover everything in 
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th« t«3ctbooks. H6 also felt that he was a better lecturer, 
because he was more Interested in the content. He was pleased to 
have more contact with students and enjoyed his role as 
discussion leader, although he felt he needed much improvement in 
this area. He was especially positive about the reading guides 
both in terms of coverage of textbook material and getting 
students to think about questions which required them to go 
beyond rote learning of information. He liked the notion of 
concept mapping and found that by incorporating it into his own 
thinking, he was better able to explain to his students the 
structure of his subject matter and some ways of thinking about 
it. He was continually surprised that so many students came to 
the tutorial sessions with reading guides prepared and concept 
maps completed or attempted. His evaluation of the teaching done 
by the graduate teaching assistants was also very positive. He 
felt this success was due to the structure and support they 
received from both the reading guides and the research team 
meetings. 

The next time he taught the same course (Fall semester, 
1991) , he continued using the reading guides with inserted 
concept mapping questions. He wanted to continue with the 
discussion sections but was thwarted by the unavailability of 
classroom space. 

Conclusions 

The primary purpose of this study was to establish the 
effectiveness of the teaching improvement process referred to as 
practice-centered inquiry. Faculty discussions groups and formal 
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classrooB research studies were the two methods used in this 
investigation, it was predicted that these methods would support 
the csntral premise of practice-centered inquiry, that a large 
part of a professor's knowledge about teaching evolves from 
reflection and experimentation. 

The content of the faculty discussions and the number of 
professors who actually tried various teaching approaches suggest 
that the faculty discussion group structure was appropriate in 
addressing the more informal levels of reflection and 
experimentation. Participants were very positive about the 
discussion groups and were able to be quite specific in 
describing their learning. A more objective and comprehensive 
method Of documenting the reflective process should be sought. 
Furthermore, the extent to which this informal reflection and 
experimentation fosters more permanent changes in thinking about 
teaching or in teaching practice has not yet been documented, 
only a few individual classroom studies have been completed to 
date, but the process which has been established holds promise in 
contributing to the investigation of classroom teaching. These 
studies are also potentially valuable as case examples for others 
studying teaching and learning in higher education. 



ERIC 



36 



35 

R«f«7«no«s 

Bloom, B. Taxonemy o f Educational Obiectivgg. H4n<abQ0K li 

<; ^panltlvc Domain . New York: David McKay Co., Inc., 1956. 
Boice, Ro, and J.L. Turner. "The FIPSE-CSULB Mentoring Project 

for New Faculty." Ia_JLa£Xaye...t:ng A<?a<a?ffy 8 (1989) :117-139. 
Cranton, P. A. 1982. •♦Evaluat;nq Complex Student Learning." Centre 

for University Teaching and Learning, McGill University. 

Photocopy in our po«5«a»ion. 
Cranton, P.Ao, and W. Hi I liAr-ner . "The Relationships Between 

Student Ratings and :-«'ructor Behavior: Implications for 

Improving Teaching. *• :r.« :M)Mi^IL-lQli30ial-O^ 

Education 11, no. i : 73-81. 

Donald, J.G. "Knowledge '.res: Methods for Exploring Course 

Content." Jm yn^X • . i * *iX__EducatLLqn 54, no. 1 (1983): 

31-41. 

Elrick, M. "Improving Ir.s' r. - . in Universities: A Case Study 
of the Ontario Univ«rs.^.es Program for Instructional 
Development." 21lg...^AiiAi^JLQ-J-0~^r.n.a 1^ oL-H iqher gdyg.a.t^lsn 20, 
no. 2 (1990) s 61-79. 

Kemp, J.E. The Instru ct lonft^ ^SLSJ^-JXfiS.SS.^l. London: Scott, 
Forettisan and Co o , 1 9 9 ^ 

Kozma, R.B. "A Grounded T-*- r, of Instructional Innovation in 
Higher Educatioii." ;;^j:r.Ai_^l-Jilgllg£-Myca^^ 56, no. 3 
(1985) S3 00- 3 19. 

Levinson-Rose, J., and R.J. Menges. "Improving College Teachings 
A Critical Review of Research*" SevLew_^l_S^city^nal 
SiaasaESH 51, no. 3 (1981) :403-434. 



er|c 37 



Mahler, S., and D.E. Benor. "Short and Long Term Effects of a 
Teacher-Training Workshop in Medical School." Higher 
Edttgfltiop 13 (1984) :265-273. 
Mahler, S., R. Hoz, d. Fischl, E. Tov-ly, and o.Z, Lernau. 
"Didactic Use of Concept Mapping in Higher Education." 
Instructional Sclenc^ 20 (1991):25-47. 
Menges, R.j., and W.C. Rando. "What are Your Assumptions? 
Improving Instruction by Examining Theories." collacye 
Tgflghinq 37, no. 2 ( 1989) : 54-60. 
Novak, J,D. "Concept Maps and Vee Diagrams: Two Metacognitive 
Tools to Facilitate Meaningful Learning." IHfiimsUflnai 
Science 19 (1990) : 29-52 . 
Okebukola, P.A., and 0. Jedga. "Cognitive Preference and Learning 
Mode as Determinants of Meaningful Learning Through Concept 
Mapping." gwng? MwcflUPP. 72, no. 2 (1988) :489-500. 
Powell, J.P., and V.s. Shanker. "The Course Planning and 

Monitoring Activities of a University Teacher." Higher 
Education n (1982 ): 289-301 . 
Resnick, L.B. h instructions I Psychology." Annual R.vio.^ r .f 

P-gyghglogy 32 (1981) -.659-704. 
Resnick, L.B. -Instruction.- In Knowing, r.^^rnim 

i ngt a nigt i ftn . edited by l. Resnick, 1-24. Hillsdale, nJ: LEA, 
1989. 

schratz, M. "Researching While Teaching: A Collaborative Action 
Research Model to Improve College Teaching." Journal r^f 
EXggllen^g in Collpge Teachingf 1 (199O) : 98-108 . 

38 



Sh«rBan, T.M. , L.P. Armistead, F. Fowler, M.A. Barksdale, and F. 

Ralf. **Tha Quest for Excellence in University Teaching." 

Journal of Higher Education 58, no. 1 (1987) : 66-84 . 
Shore, B.M., S. Pinker, and M. Bates. "Research as a Model for 

University Teaching." Higher Education 19, no. 1 

(1990) :21-35. 

Shulman, L.s. "Knowledge and Teaching: Foundations of the New 

Reform." Harvard E ducational Review 57(1) (1986):l-22. 
Shulnan, L.S. "Paradigms and Research Programs in the Study of 

Teaching: A Contemporary Perspective." In Handbook on 

Research on Teaching , edited by M.c. Wittrock, 3-36. 

New York: Macmillan, 1986. 
Tiberius, R.G., H.D. Sackin, J.M. Slingerl?.?* . Jubas, M. Bell, 

and A. Mat low. "The Influence of Stuov . - Evaluative Feedback 

on the Improvement of Clinical Teaching." Journal of Higher 

Education 60, no. 6 (1989) : 665-680. 
Van Note Chism, N. , and D. Sanders. "The Place of 

Practice-centered Inquiry in a Faculty Development Program." 

To Improve the Academv 5 ( 1986) : 56-64 . 
Wentzel, H.K. "Seminars in College Teaching: An Approach to 

Faculty Development." College Teaching 35, no. 2 

(1987) :70-71. 

Weston, C, and P. A. Cranton. "Selecting Instructional 
Strategies." Journal of Higher E ducation 57, no. 3 
(1986) :259-288. 



Wilson, R.C. "Improving Faculty Teaching; Effective Use of 
Student Evaluations and Consultants." Journal of Hieyhar 
Education 57, no. 2 (1986) : 196>211. 



10 



9 



39 



Table 1 



Mean Pre- and Post- 


-test scores 
Treaumenu 


of Conceptual Knowledge by 
Level 






Test 


Scores 


A a4*m am4* 


n 


Pre 


Post 


RdaQin^ Guide 


45 










6.15 
(3.53) 


12.63 
(5.62) 


Reading Guide With 
Concept Mapping 


36 






If 

SB 




10.23 
(4.79) 


13.97 
(5.43) 


Reading Guide with 
Concept Mapping and 
Immediate Feedback 


43 










8.70 
(3.70) 


11.74 
(4.64) 



Note: H represents Mean Scores 

^ represents standard Deviation 
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Table 2 

Percentage of Misconceptions on Pre- and Post- tests 

For Each Treatment Level 



Tests 



Reading Guide with 4 3 

Concept Mapping and 
Immediate Feedback 



Treatment q pre Post 

Reading Guide 4 5 

E 17.14% 14.49% 

Reading Guide with 3 6 

Concept Mapping 



16.75% 12.25% 



19.17% 8.13% 
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Figure 1. A concept map representing a Politicai Science Course 
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Figure 2. Pre- and post-test differences of conceptual knowledge. 
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Figure 3. Pre- and post-test differences in percentage of misconceptions. 
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